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Abstract. We consider two min-max problems: (1) minimizing the supre- 
mum of finitely many rational functions over a compact basic semi-algebraic 
set and (2) solving a 2-player zero-sum polynomial game in randomized strate- 
gies with compact basic semi-algebraic sets of pure strategies. In both problems 
the optimal value can be approximated by solving a hierarchy of semidefinite 
relaxations, in the spirit of the moment approach developed in Lasserre I24II26I . 
This provides a unified approach and a class of algorithms to compute Nash 
equilibria and min-max strategies of several static and dynamic games. Each 
semidefinite relaxation can be solved in time which is polynomial in its input 
size and practice on a sample of experiments reveals that few relaxations are 
needed for a good approximation (and sometimes even for finite convergence), 
a behavior similar to what was observed in polynomial optimization. 



1. Introduction 

Initially, this paper was motivated by developing a unified methodology for solv- 
ing several types of (neither necessarily finite nor zero-sum) A^-player games. But 
it is also of self-interest in optimization for minimizing the maximum of finitely 
many rational functions (whence min-max) on a compact basic semi-algebraic set. 
Briefly, the moment-s.o.s. approach developed in [211 126] is extended to two large 
classes of min-max problems. This allows to obtain a new numerical scheme based 
on semidefinite programming to compute approximately and sometimes exactly (1) 
Nash equilibria and the min-max of any finite gamqj and (2) the value and the 
optimal strategies of any polynomial two-player zero-sum gameo. In particular, the 
approach can be applied to the so-called Loomis games defined in [32j and to some 
dynamic games described in Kolhberg [21j . 

Background. Nash equilibrium [33] is a central concept in non-cooperative game 
theory. It is a profile of mixed strategies (a strategy for each player) such that each 
player is best-responding to the strategies of the opponents. An important problem 
is to compute numerically a Nash equilibrium, an approximate Nash equilibrium 
for a given precision or all Nash equilibria in mixed strategies of a finite game. 
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^Games with finitely many players where the set of pure actions of each player is finite. 

■^Games with two players where the set of pure actions of each player is a compact basic 
semi-algebraic-set and the payoff function is polynomial. 
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It is well known that any two-player zero-sum finite game (in mixed strategies) is 
reducible to a linear program (Dantzig [Sj) and hence equilibria could be computed 
in polynomial time (Khachiyan 20]). 

Lemke and Howson ,29j provided a famous algorithm that computes a Nash- 
equilibriuni (in mixed strategies) of any 2-player non-zero-sum finite game. The 
algorithm has been extended to A^-player finite games by Rosenmiiller [39] , Wilson 
[50j and Govindan and Wilson [12] . 

An alternative to the Lemke-Howson algorithm for 2-player games is provided in 
van den Elzen and Talman ^ and has been extended to n-player games by Herings 
and van den Elzen [16] . As shown in the recent survey of Herings and Peeters 
[17] ■ all these algorithms (including the Lemke-Howson) are homotopy-based and 
converge (only) when the game is non-degenerate. 

Recently, Savani and von Stengel [41] proved that the Lemke-Howson algorithm 
for 2-player games may be exponential. One may expect that this result extends to 
all known homotopy methods. Daskalakis, Goldberg and Papadimitriou 4J proved 
that solving numerically 3-player finite games is harcil. The result has been ex- 
tended to 2-player finite games by Chen and Deng [6 . Hence, computing a Nash 
equilibrium is as hard as finding a Brouwer- fixed point. For a recent and deep sur- 
vey on the complexity of Nash equilibria see Papadimitriou [35j . For the complexity 
of computing equilibria on game theory in general, see |34| and [40] . 

A different approach to solve the problem is to view the set of Nash equilibria as 
the set of real nonnegative solutions to a system of polynomial equations. Methods 
of computational algebra (e.g. using Grobner bases) can be applied as suggested 
and studied in e.g. Datta pi, Lipton [SfT and Sturmfels [47|. However, in this 
approach, one first computes all complex solutions to sort out all real nonnegative 
solutions afterwards. Interestingly, polynomial equations can also be solved via 
homotopy-based methods (see e.g. Verschelde (48|). 

Another important concept in game theory is the min-max payoff of some player 
i, v_i- This is the level at which the team of players (other than i) can punish 
player i. The concept plays an important role in repeated games and the famous 
folk theorem of Aumann and Shapley [2]. To our knowledge, no algorithm in 
the literature deals with this problem. However, it has been proved recently that 
computing the min-max for 3 or more player games is NP-hard [J . 

The algorithms described above concern finite games. In the class of polynomial 
games introduced by Dresher, Karlin and Shapley (1950), the set of pure strategies 
5'* of player i is a product of real intervals and the payoff function of each player 
is polynomial. When the game is zero-sum and 5* = [0, 1] for each player i = 
1,2, Parrilo [37] showed that finding an optimal solution is equivalent to solving 
a single semidefinite program. In the same framework but with several players. 
Stein, Parrilo and Ozdaglar ^46j propose several algorithms to compute correlated 
equilibria, one among them using SDP relaxations. Shah and Parrilo [H] extended 
the methodology in [37] to discounted zero-sum stochastic games in which the 
transition is controlled by one of the two players. Finally, it is worth noticing recent 
algorithms designed to solve some specific classes of infinite games (not necessarily 
polynomial). For instance, Giirkan and Pang [13] . 



■'More precisely, it is complete in the PPAD class of all search problems that are guaranteed 
to exist by means of a direct graph argument. This class was introduced by Papadimitriou in 1361 
and is between P and NP. 
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Contribution. In the first part we consider what we call the MRF problem which 
consists of minimizing the supremum of finitely many rational functions over a com- 
pact basic semi-algebraic set. In the spirit of the moment approach developed in 
Lasserre |24[ for polynomial optimization, we define a hierarchy of semidefi- 
nite relaxations (in short SDP relaxations). Each SDP relaxation is a semidefinite 
program which, up to arbitrary (but fixed) precision, can be solved in polynomial 
time and the monotone sequence of optimal values associated with the hierarchy 
converges to the optimal value of MRF. Sometimes the convergence is finite and a 
sufficient condition permits to detect whether a certain relaxation in the hierarchy 
is exact (i.e. provides the optimal value), and to extract optimal solutions. It is 
shown that computing the min-max or a Nash equilibrium in mixed strategies for 
static finite games or dynamic finite absorbing games reduces to solving an MRF 
problem. For zero-sum finite games in mixed strategies the hierarchy of SDP re- 
laxations for the associated MRF reduces to the first one of the hierarchy, which 
in turn reduces to a linear program. This is in support of the claim that the above 
MRF formulation is a natural extension to the non linear case of the well-known 
LP-approach [5] as it reduces to the latter for finite zero-sum games. In addition, if 
the SDP solver uses primal-dual interior points methods and if the convergence is 
finite then the algorithm returns all Nash equilibria (if of course there are finitely 
many) . 

To compute all Nash equilibria, homotopy algorithms are developed in Kostreva 
and Kinard |22| and Herings and Peeters [18^ . They apply numerical techniques to 
obtain all solutions of a system of polynomial equations. 

In the second part, we consider general 2-player zero-sum polynomial games 
in mixed strategies (whose action sets are basic compact semi-algebraic sets of 
M" and payoff functions are polynomials). We show that the common value of 
max-min and min-max problems can be approximated as closely as desired, again 
by solving a certain hierarchy of SDP relaxations. Moreover, if a certain rank 
condition is satisfied at an optimal solution of some relaxation of the hierarchy, 
then this relaxation is exact and one may extract optimal strategies. Interestingly 
and not surprisingly, as this hierarchy is derived from a min-max problem over 
sets of measures, it is a subtle combination of moment and sums of squares (s.o.s.) 
constraints whereas the hierarchy for polynomial optimization is based on either 
moments (primal formulation) or s.o.s. (dual formulation) but not both. This is a 
multivariate extension of Parrilo's |37| result for the univariate case where one needs 
to solve a single semidefinite program (as opposed to a hierarchy). The approach 
may be extended to dynamic absorbing gamef0 with discounted rewards, and in 
the univariate case, one can construct a polynomial time algorithm that combines 
a binary search with a semidefinite program. 

Hence the first main contribution is to formulate several game problems as a par- 
ticular instance of the MRF problem while the second main contribution extends 
the momcnt-s.o.s. approach of [24l [26] in two directions. The first extension ( the 
MRF problem) is essentially a non trivial adaptation of Lasserre's approach [24] 
to the problem of minimizing the sup of finitely many rational functions. Notice 
that the sup of finitely many rational functions is not a rational function. However 
one reduces the initial problem to that of minimizing a single rational function 



In dynamic absorbing games, transitions are controlled by both players, in contrast with 
Parrilo and Shah j44) where only one player controls the transitions. 
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(but now of n + 1 variables) on an appropriate set of M"+^. As such, this can also 
be viewed as an extension of Jibetean and De Klerk's result }T9] for minimizing 
a single rational function. The second extension generalizes (to the multivariate 
case) Parrilo's approach j37| for the univariate case, and provides a hierarchy of 
mixed moment-s.o.s. SDP relaxations. The proof of convergence is delicate as one 
has to consider simultaneously moment constraints as well as s.o.s. -representation 
of positive polynomials. (In particular, and in contrast to polynomial optimization, 
the converging sequence of optimal values associated with the hierarchy of SDP 
relaxations is not monotone anymore.) 

To conclude, within the game theory community the rather negative compu- 
tational complexity results ([3], [4], [6], [40], [41]) have reinforced the idea that 
solving a game is computationally hard. On a more positive tone our contribution 
provides a unified semidcfinite programming approach to many game problems. It 
shows that optimal value and optimal strategies can be approximated as closely 
as desired (and sometimes obtained exactly) by solving a hierarchy of semidefinite 
relaxations, very much in the spirit of the moment approach described in [24^ for 
solving polynomial minimization problems (a particular instance of the Generalized 
Problem of Moments p6]). Moreover, the methodology is consistent with previous 
results of |5] and |37| as it reduces to a linear program for finite zero-sum games 
and to a single semidefinite program for univariate polynomial zero-sum games. 

Finally, even if practice from problems in polynomial optimization seems to 
reveal that this approach is efhcicnt, of course the size of the semidefinite relaxations 
grows rapidly with the initial problem size. Therefore, in view of the present status 
of public SDP solvers available, its application is limited to small and medium size 
problems so far. 

Quoting Papadimitriou |35| : "The PPAD- completeness of Nash suggests that any 
approach to finding Nash equilibria that aspires to be efficient [. . .] should explicitly 
take advantage of computationally beneficial special properties of the game in hand^^ . 
Hence to make our algorithm efhcient for larger size problems, one could exploit 
possible sparsity and regularities often present in the data (which will be the case 
if the game is the normal form of an extensive form game). Indeed specific SDP 
relaxations for minimization problems that exploit sparsity efficiently have been 
provided in Kojima et al. A9\ and their convergence has been proved in [25] under 
some condition on the sparsity pattern. 

2. Notation and preliminary results 

2.1. Notation and definitions. Let R[a;] be the ring of real polynomials in the 
variables x — (xi, . . . ,a;„) and let (X")Q,gN be its canonical basis of monomials. 
Denote by T,[x] C M.[x] the subset (cone) of polynomials that are sums of squares 
(s.o.s.), and by M.[x]d the space of polynomials of degree at most d. Finally let |ja;|| 
denote the Euclidean norm of x G M". 

With y =: (ya) C M being a sequence indexed in the canonical monomial basis 
(X"), let Ly : M.[x] ^ M be the linear functional 

/ (= E ^ E / e i^N- 

Moment matrix. Given y — (ya) C M, the moment matrix Md{y) of order d 
associated with y, has its rows and columns indexed by {x") and its (a, /3)-entry is 
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defined by: 

Md(y)(a,/3) = Vc.+p. \a\, < d. 

Localizing matrix. Similarly, given y = (y^) C M and 9 £ M.[x\ {— ^^d-yX^), 
the localizing matrix il/jj(0,y) of order d associated with y and 9, has its rows and 
columns indexed by (a;") and its (a,/3)-entry is defined by: 

Md{9,y){a,(i) := Ly{x-^+P9{x)) = Y,S^Vi+c.+0. WP\ < d- 

7 

One says that y = (ya) C M has a representing measure supported on K if there is 
some finite Borel measure on K such that 

Va = I x"dii{x), Va e N". 

For later use, write 

(2.1) Afd(y) = ^ y„Bc. 

(2.2) Ma{9,y) = ^ y^B^, 

aGN" 

for real symmetric matrices {Ba,B^) of appropriate dimensions. Note that the 
above two summations contain only finitely many terms. 

Definition 2.1 (Putinar's property). Let {gj)™^i C M\x\. A basic closed semi 
algebraic set K := {x G R" : gj{x) > 0, : j = 1, . . . , m} satisfies Putinar's property 
if there exists u G M\x\ such that {x : u{x) > 0} is compact and 

m 

(2.3) u = (To + ^<Jj gj 

for some s.o.s. polynomials {ajYjLg C Equivalently, for some M > the 

quadratic polynomial x ^ M — has Putinar's representation Ii2.3\) . 

Obviously Putinar's property implies compactness of K. However, notice that 
Putinar's property is not geometric but algebraic as it is related to the representa- 
tion of K by the defining polynomials (.gj)'s. Putinar's property holds if e.g. the 
level set {x : gj{x) > 0} is compact for some j, or if all gj are affine and K is com- 
pact (in which case it is a polytope). In case it is not satisfied and if for some known 
M > 0, ||a;|p < M whenever x € K, then it suffices to add the redundant quadratic 
constraint gm+i{x) ■= M — ||a;|p > to the definition of K. The importance of 
Putinar's property stems from the following result: 

Theorem 2.2 (Putinar [38 ). Let {gj)^i C M.[x] and assume that 

K:={.xeE" :gj(a;) >0, j = l,...,m} 

satisfies Putinar's property. 

(a) Let / G IR[a;] be strictly positive on K. Then f can be written as u in i2.3\) . 

(b) Let y = [tja)- Then y has a representing measure on K if and only if 

(2.4) Mrf(y)^0, A/d(g„y)^0, j = l,...,m; d = 0,l,... 
We also have: 
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Lemma 2.3. Let K C M" be compact and let p, q continuous with g > on K. Let 
M(K) be the set of finite Borel measures on K and let P(K.) C Af (K) be its subset 
of probability measures on K. Then 



(2.5) 



Jf^pd^ . , [ f 

min = min { p dip : / q dip = 1 } 



(2.0) = mm / — = mm ■ 



AieP(K) Jk 9 (/(x) 
Proof. Let p* min2;{p(.T)/g(a;) : x G K}. As i? > on K, 

J^pd^i _ J^{p/q)qd^i 



>P* 



Hence ii fi G ^(K) then Jj^{p/q)d^ > p* J^d^ = p* . On the other hand, with 
X* S K a global minimizer oip/q on K, let p :— S^' G P{^) be the Dirac measure 
at a; = X*. Then J^^pdp/ Jj^ qdp = p{x* ) / q{x*) — Jj^{p/q)dp = p* , and therefore 

/kPc^M ■ [ Pa ■ P(^) 



Me-P(K) Jj^ gd^ /jspCk) 7k 9 ^^^^^ 9(2^) 

Next, for every Lp G Af(K) with J^^qdip — 1, Jj^pdip > Jj^p*qdip = p* , and so 
min(pgA/(K){/KP'^V' : IkI^ip = 1} > p* . Finally taking ip q{x*y^dx' yields 
Jj^qdp = 1 and Jj^pdip ^ p{x*)/q{x*) = p*. 

Another way to see why this is true is throughout the following argument. The 

function fi is quasi-concave (and also quasi-convex) so that the optimal 

value of the minimization problem is achieved on the boundary. □ 

3. Minimizing the max of finitely many rational functions 
Let K C M" be the basic semi- algebraic set 

(3.1) K := {a;eR" :gj(x) >0, j = l,...,p} 

for some polynomials {gj) C M.[x], and let fi — Pi/qt be rational functions, i = 
0, 1, . . . , m, with Pi, qi £ W[x]. We assume that: 

• K satisfies Putinar's property (see Definition 12. ip and, 

• (7i > on K for every i — 0, . . . ,m. 

Consider the following problem denoted by MRF: 

(3.2) MRF : p min{/o(a;) -I- max fi{x) -.xgK}, 

X i— l,...,m 

or, equivalently, 

(3.3) MRF: p min{/o(a;) + x : z > /^(a;), i = 1, . . . , m; x G K }. 

With K C M" as in ([3T|) . let K C be the basic semi algebraic set 

(3.4) K := {{x,z) G K" X M : a; G K; zqi{x) ~ p,{x) > 0, i = 1, . . . , to} 
and consider the new infinite-dimensional optimization problem 

(3.5) V : p := min{ / [pa + z q^) dp : / qodp = l, p G M(K)} 

Jk. 7k 
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where M(K) is the set of finite Borel measures on K. Problem ()3.5|) is a particular 
instance of the Generalized Problem of Moments for which a general methodology 
(based on a hierarchy of semidefinite relaxations) has been described in '26]. To 
make the paper self-contained we explain below how to apply this methodology in 
the above specific context. 

Proposition 3.1. Let K C M" be as in iTO]) . Then p. 
Proof. The following upper and lower bounds 

z := min min/i(a;); 'z :— max max/i(x), 

2=1,. ...m 2;GK i=l,...,m a:;GK 

are both well-defined since K is compact and > on K for every i = 1, . . . ,m. 
Including the additional constraint z < z < z in the definition of K makes it 
compact without changing the value of p. Next observe that 

■ f Pojx) + zqo{x) ^ 
(3.6) p = mm < — : (x, z) £ K 



(x,z) { qo{x) 

Applying Lemma [2 . 31 with K in lieu of K, and with (x, z) i— s- p(x, z) :— ]iii(x)+ zqoix) 
and [x, z) ^ q(x, z) := qa{x), yields the desired result. □ 

Remark 3.2. (a) When rn ~ 0, that is when one wishes to minimize the rational 
function /q on K, then V reads min^lj^, fodp : qodp — 1] p E Af (K)}. Using 
the dual problem V* : max{z : ^0(2;) — zq^^x) > Va; G K}, Jibetean and 
DeKlerk [19j proposed to approximate the optimal value by solving a hierarchy of 
semidefinite relaxations. The case m > 2 is a nontrivial extension of the case m = 
because one now wishes to minimize on K a function which is not a rational function. 
However, by adding an extra variable z, one obtains the moment problem (j3.5p . 



which indeed is the same as minimizing the rational function {x, z) i— > 
on a domain K C R""*"^. And the dual of V now reads max{p : po(a^) + zq[){x) — 
PQo{x) > V(x, z) S K}. Hence if K is compact and satisfies Putinar's property 
one may use the hierarchy of semidefinite relaxations defined in |19| and adapted 
to this specific context; see also [26] . 

(b) The case m = 1 (i.e., when one wants to minimize the sum of rational 
functions /o + /i on K) is also interesting. One way is to reduce to the same 
denominator qo x qi and minimize the rational function (po9i + Pi'?o)/'7o9i- But 
then the first SDP relaxation of [26l would have to consider polynomials of 
degree at least d := max[degpo + deg(7i,degpi -I- deg go , deg go + deggi], which 
may be very penalizing when qo and qi have large degree (and sometimes it may 
even be impossible!). Indeed this SDP relaxation has as many as 0{n'^) variables 
and a linear matrix inequality of size 0{n^'^/^'^). In contrast, by proceeding as 
above in introducing the additional variable z, one now minimizes the rational 
function (po + zqQ)/qQ which may be highly preferable since the first relaxation 
only considers polynomials of degree bounded by max[degpo7 1 + deggo] (but now 
in n -|- 1 variables). For instance, if degpi = degqi = v, i — 1,2, then one has 
0{n'"~^^) variables instead of 0{n^") variables in the former approach. 



We next describe how to solve the MRF problem via a hierarchy of semidefinite 
relaxations. 
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SDP relaixations for solving the MRF problem. As K is compact and qi > 
on K, for all i, let 



(3.7) Ml : 
and 

(3.8) M2 
Redefine the set K to be 



ma.x{\pi{x)\,x £ K} 
=i,...,m [ mm{qi(x),x G K} 

miTi{pi(x) , X G K} 



i,...,m \ max{qi{x), X G K} 



(3.9) 


K := {{x, 


z) G W 


' X R : hj{x,z) > 0, j = 


1, 


. . .p + m + 1} 


with 














( {x,z) l-> 


hj{x 




j 


= 1, . . . ,p 


(3.10) 




hj{x 


z) := zqjix) -Pj{x) 


j 


= p + 1, . . . ,p + m 




[ {x,z) ^ 


hj{x 


z) := (Mi-z)(z-Af2) 


j 


= m + p + 1 


Lemma 


3.3. Let K 


C M" 


satisfy Putinar's property 




Then the set K C 



defined in \3. ^j) satisfies Putinar's property. 

Proof. Since K satisfies Putinar's property, equivalently, the quadratic polynomial 
X I— > u{x) := M — \\x\\^ can be written in the form (|2.3p . i.e., u[x) = ao[x) + 
Y^^=i '^ji^)9ji^) fo'^ some s.o.s. polynomials (ctj) C Next, consider the 

quadratic polynomial 

{x,z) w{x,z) = M - ll^lp + {Ml - z){z - M2). 

Obviously, its level set {x : w{x, z) > 0} C M"+^ is compact and moreover, w can 
be written in the form 

p 

wix,z) = ao{x)+^aj{x)gj{x) + {Mi-z){z-M2) 

m+p+1 

= cr'o{x,z)+ ^ aj{x,z)hj{x,z) 

for appropriate s.o.s. polynomials {a'j) C S[a;,2]. Therefore K satisfies Putinar's 
property in Definition 12. li the desired result. □ 

We are now in position to define the hierarchy of semidefinite relaxations for solv- 
ing the MRF problem. Let y = (j/q,) be a real sequence indexed in the monomial 
basis {x'^z'') of R[x, z] (with a = {l3, fc) G x N). 

Let ho{x, z) :— pq(x) + zq^ix), and let Vj := \{deghj)/2~\ for every j = 0, . . . , m + 
p + 1. For r > To max Vj, introduce the hierarchy of semidefinite pro- 

^— 0, . . . ,p+m+l 

grams: 

min Ly[ho) 

(3.11) Q.J s-t- ^r{y) hO 

Mr-^^(hj,y) hO, j ^l,...,m + p+l 

Ly{qo) = 1, 

with optimal value denoted inf (and min if the infimum is attained) . 
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Theorem 3.4. Let K C M" (compact) be as in 113.1]} . Let Qr be the semidefinite 
program iTOT]) with (hj) C R[x,z] and Mi,M2 defined in IgJOj) and [3l\)- (3^) 
respectively. Then: 

( a) inf Qr t P as r ^ oo. 

(b) Let be an optimal solution of the SDP relaxation in iS.ll]) . If 
(3.12) rankMr(y'') = rankMr_ro(y'') = t 

then min Q,. = p and one may extract t points {x* {k))l.^-^ C K, all global minimizers 
of the MRF problem. 

(c) Let be a nearly optimal solution of the SDP relaxation iS.ll]) (with say 
inf Qr < Ly'^ < inf Qr + ^/r. If US. 6\) has a unique global minimizer x* G K then 
the vector of first- order moments (Lyr(xi), . . . , Lyr(a;„)) converges to x* as r —>■ oo. 

Proof. As already mentioned in Remark l3.2i convergence of the dual of the semidef- 
inite relaxations (jS.lip was first proved in Jibetean and de Klerk |l9j for minimizing 
a rational function on a basic compact semi-algebraic set (in our context, for mini- 
mizing the rational function {x, z) ^ {po{x) + zqo{x)) / qo{x) on the set K C M"+^). 
See also [261 §4-1] and [28l Theor. 3.2]. In particular to get (b) see [28l Theor. 3.4]. 
The proof of (c) is easily adapted from Schweighofer [13]. □ 

Remark 3.5. Hence, by Theorem 13. 4r b). when finite convergence occurs one may 
extract t := rankAfr(y) global minimizers. On the other hand, a generic MRF 
problem has a unique global minimizer G K and in this case, even when the 
convergence is only asymptotic, one may still obtain an approximation of x* (as 
closely as desired) from the vector of first-order moments {Lyr{xi), . . . ,iyr(x„)). 
For instance, one way to have a unique global minimizer is to e-perturb the objec- 
tive function of the MRF problem by some randomly generated polynomial of a 
sufficiently large degree, or to slightly perturb the coefficients of the data {hi,gj) 
of the MRF problem. 

To solve p. Ill) one may use e.g. the Matlab based public software GloptiPoly 
3 [15] dedicated to solve the generalized problem of moments described in [26]. It 
is an extension of GloptiPoly [14] previously dedicated to solve polynomial opti- 
mization problems. A procedure for extracting optimal solutions is implemented 
in Gloptipoly when the rank condition (|3.12p is satisfiecH. For more details the 
interested reader is referred to [15] and www. laas . f r/^henrion/sof tware/. 

Remark 3.6. li gj is affine for every j = 1, . . . ,p and if pj is affine and qj = 1 for 
every j = 0, . . . , m, then hj is affine for every j — 0, . . . ,m. One may also replace 
the single quadratic constraint hm+p+i 

{x,z) = (Ml - z){z - M2) > with the 
two equivalent linear constraints hm+p+iix, z) = Mi — z > and hm+p+2{x, z) — 
z — M2 > 0. In this case, it suffices to solve the single semidefinite relaxation Qi, 



In fact GloptiPoly 3 extracts all solutions because most SDP solvers that one may call in 
GloptiPoly 3 (e.g. SeDuMi) use primal-dual interior points methods with the self-dual embedding 
technique which find an optimal solution in the relative interior of the set of optimal solutions; 
see |27l §4.4.1, p. 663]. In the present context of Il3.11| l this means that at an optimal solution 
y*, the moment matrix Mr{y*) has maximum rank and its rank corresponds to the number of 
solutions. 
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which is in fact a hnear program. Indeed, for r = 1, y = (j/o, (a;, z), Y) and 

2/0 I {x z) 



Then (fXTTI) reads 



Qi 



Mi(y) = 



min /lo(a^) 
y 

s.t. 



Mi{y) 
hj{x,z] 
2/0 



^ 
>0, 
= 1. 



p + 2 



As Vj = 1 for every j, Mi„i(/ij,y) ^ <^ Mo(/ij,y) = Ly{hj) = hj{x,z) > 0, a 
hnear constraint. Hence the constraint Mi(y) ^ can be discarded as given any 
{x,z) one may always find Y such that Afi(y) >^ 0. Therefore, (|3.1ip is a hnear 
program. This is fortunate for finite zero-sum games apphcations since computing 
the value is equivalent to minimizing a maximum of finitely many linear functions 
(and it is already known that it can be solved by Linear Programming) . 



4. Applications to finite games 

In this section we show that several solution concepts of static and dynamic finite 
games reduce to solving the MRF problem (|3.2p . Those are just examples and one 
expects that such a reduction also holds in a much larger class of games (when they 
are described by finitely many scalars). 

4.1. Standard static games. A finite game is a tuple (A^, {5*} a? ' {5*}i-i jv) 
where A^ e N is the set of players, is the finite set of pure strategies of player i 
and : S ^ M is the payoff function of player i, where S 5^ x ... x . The set 

A' = I (P^(s')),.g5. : f{sl > 0, 5] f{s^) ^ l| 
I s'es- ) 

of probability distributions over 5* is called the set of mixed strategies of player i. 
Notice that A* is a compact basic semi- algebraic set. If each player j chooses the 
mixed strategy p'(-), the vector denoted p — (p^, ■■■,p^) £ A : = x ... x A^ is 
called a profile of mixed strategies and the expected payoff of a player i is 

s=(s1,...,s")gS 

This is nothing but the multi- linear extension of g*. For a player i, and a 
profile p, let be the profile of the other players except i: that is p^* — 
(pi, ...,p'-\p'+\ Let = X ... X S'*-i X S'+^ x ... x and define 

g\s\p-^) = ^ - ^ P'^'i^'^') X X - X p'^(s'^)ffXs), 

where s"* := {s\ s'-\ s'+\ s^) e S-\ 
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A profile po is a Nash [33] equilibrium if and only for all i = 1,...,N and all 
e S'*, g^{po) > 5*(s%Po*) or equivalently if: 



(4.1) Po e argmin \ max max {g\s\po ') - g\po)} 



Since each finite game admits at least one Nash equilibrium [33] . the optimal 
value of the min-max problem (|4.ip is zero. Notice that (|4.ip is a particular instance 
of the MRF problem (|3.2p (with a set K = A that satisfies Putinar's property 
and with qi = 1 for every i = 0, ...,to), and so Theorem 13.41 applies. Finally, 
observe that the number ni of polynomials in the inner double "max" of (|4.ip (or, 
equivalently, m in (|3.2p ) is just m — X]r=i \^^\^ i-'^-' J^^^ ^he total number of 
all possible actions. 

Hence by solving the hierarchy of SDP relaxations p. lip , one can approximate 
the value of the min-max problem as closely as desired. In addition, if (|3.12p is 
satisfied at some relaxation Q^, then one may extract all the Nash equilibria of the 
game. 

If there is a unique equilibrium p* then by Theorem l3.4r c). one may obtain a so- 
lution arbitrary close to p* and so obtain an e-equilibrium in finite time. Since game 
problems are not generic MRF problems, they have potentially several equilibria 
which are all global minimizers of the associated MRF problem. Also, perturbing 
the data of a finite game still leads to a non generic associated MRF problem 
with possibly multiple solutions. However, as in Remark 1 3. 5) one could perturb the 
MRF problem associated with the original game problem to obtain (generically) 
an e-perturbcd MRF problem with a unique optimal solution. Notice that the 
e-perturbed MRF is not necessarily coming from a finite game. Doing so, by The- 
orem [3311c), one obtains a sequence that converges asymptotically (and sometimes 
in finitely many steps) to an e-equilibrium of the game problem. Recently, Lipton, 
Markakis and Mehta |31j provided an algorithm that computes an e-equilibrium 

in less than exponential time but still not polynomial (namely n where n is 
the total number of strategies). This promising result yields Papadimitriou [35] to 
argue that ^^finding a mixed Nash equilibrium is PPAD-complete raises some inter- 
esting questions regarding the usefulness of Nash equilibrium, and helps focus our 
interest in alternative notions (most interesting among them the approximate Nash 
equilibrium)" . 

Example 4.1. Consider the simple illustrative example of a 2 x 2 game with data 



si 

si (0,0) 



(a,c) 



si 
(0,0) 
(6,d) 



for some scalars {a,b,c,d). Denote xi £ [0, 1] the probability for player 1 of playing s\ 
and a;2 G [0, 1] the probability for player 2 of playing s\. Then one must solve 



mm max < 

2:1,3:26 [0,1] 



ax\ — axiX2 — b{l — 2:i)(l — X2) 

b{l — X2) — axiX2 — b{l — xi){l — X2) 

cxi — CX1X2 — d{l — xi){l — X2) 

d(l — xi) — CX1X2 — d{l — a;i)(l — X2) 
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We have solved the hierarchy of semidefinite programs (I3.11|l with GloptiPoly 3 [15] . For 
instance, the moment matrix M\{y) of the first SDP relaxation Qi reads 



Mi(y) = 



yo 


yioo 


yoio 


yool 


ywo 


2/200 


yoio 


yooi 


J/OIO 


J/llO 


J/020 


you 


2/001 


yioi 


J/Oll 


2/002 



and Qi reads 

min yooi 
y 

s.t. A/i(y) >z 

i/ooi - ayioo + ayuo + &(2/o - 2/ioo - j/oio + yno) > 
. j i/ooi - hyo + fei/oio + aj/iio + 6(j/o - J/ioo - yoio + yno) > 

yooi - ci/ioo + Cl/lio + rf(?/o - i/ioo - j/oio + ?/iio) > 
2/00 - dj/o + rfj/ioo + cj/iio + d(yo — j/ioo - yoio + yiio) > 
yioo - 2/200 > 0; i/oio - 2/020 > 0; 9 - 2/002 > 
2/0=1 

With (a,fe,c,d) = (0.05,0.82,0.56,0.76), solving Q3 yields the optimal value 3.93.10"" 
and the three optimal solutions (0,0), (1, 1) and (0.57575, 0.94253). With randomly gen- 
erated a, b,c,d £ [0, 1] we also obtained a very good approximation of the global optimum 
and 3 optimal solutions in most cases with r = 3 (i.e. with moments or order 6 only) 
and sometimes r = 4. 

We have also solved 2-player non-zero-sum pxq games with randomly generated reward 
matrices A,B £ R*"^' and p,q < 5. We could solve (5,2) and (4,5) (with g < 3) games 
exactly with the 4th (sometimes 3rd) SDP relaxation, i.e. inf Q4 = 0(10"^") ~ and 
one extracts an optimal solution 0. However, the size is relatively large and one is close 
to the limit of present public SDP solvers like SeDuMi. Indeed, for a 2-player (5, 2) or 
(4,3) game, Q3 has 923 variables and M3(y) £ R^'*'^*'', whereas Q4 has 3002 variables 
and M4(y) G r2iox2io^ p^^. 4-, ^^^^^ y^^^ -^j-^^ variables and M3(y) G R120X120 
and Q3 is stiU solvable, whereas Q4 has 6434 variables and M4(y) G k330x330 

Finally we have also solved randomly generated instances of 3-player non-zero sum 
games with (2, 2, 2) actions and (3, 3, 2) actions. In all (2, 2, 2) cases the 4th relaxation Q4 
provided the optimal value and the rank-test (|3.12p was passed (hence allowing to extract 
global minimizers). For the (3, 3, 2) games, the third relaxation Q3 was enough in 30% of 
cases and the fourth relaxation Q4 in 80% of cases. 

Another important concept in game theory is the min-max payoff which plays 
an important role in repeated games (Aumann and Shapley [2]): 



Uj = min max(/*(s',p *) 

where A"' = x ... x A'"^ x A'+^ x ... x A^. This is again a particular instance of 
the MRF problem (I3.2|l . Hence, it seems more difficult to compute the approximate 
min-max strategies compared to approximate Nash equilibrium strategies because 
we do not know in advance the value of v^ while we know that the min-max value 
associated to the Nash problem is always zero. This is not surprising: in theory, 
computing a Nash-equilibrim is PPAD-complete |35j while computing the min-max 

^In general, it is not known which relaxation suffices to solve the min-max problem. Also, as 
already mentioned, GloptiPoly 3 extracts all solutions because most SDP solvers that one may 
call in GloptiPoly 3 (e.g. SeDuMi) use primal-dual interior points methods with the self-dual 
embedding technique which find an optimal solution in the relative interior of the set of optimal 
solutions. This is explained in [271 §4.4.1, p. 663] 
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is NP-hard [3 . In the case of two players, the function g*(s',p is hnear in p 
By remark [3.61 it suffices to solve the first relaxation Qi, a linear program. 



Remark 4.2. The Nash equilibrium problem may be reduced to solving a system 
of polynomial equations (see e.g. [5]). In the same spirit, an alternative for the 
Nash-equilibrium problem (but not for the MRF problem in general) is to apply 
the moment approach described in Lasserre et al. [27j for finding real roots of poly- 
nomial equations. If there are finitely many Nash equilibria then its convergence 
is finite and in contrast with the algebraic methods [H I30|, I47j mentioned above, it 
provides all real solutions without computing all complex roots. 

4.2. Loomis games. Loomis |35] extended the min-max theorem to zero-sum 
games with a rational fraction. His model may be extended to A^-player games 
as follows. Our extension is justified by the next section. 

Associated with each player i € N are two functions 5* : S ^ R and /* : S ^ M 
where /* > and S := x ... x . With same notation as in the last section, let 
their multi-linear extension to A still be denoted by and /\ That is, for p e A, 
let: 

s=(si,...,s«)es 
and similarly for /*. 

Definition 4.3. A Loomis game is defined as follows. The strategy set of player i 
is A* and if the profile p £ ^ is chosen, his payoff function is h^{p) — jr^- 

One can show the following new lemmgQ. 
Lemma 4.4. A Loomis game admits a Nash equilibriun^. 

Proof. Note that each payoff function is quasi-concave in (and also quasi-convex 
so that it is a quasi- linear function). Actually, if h''{p\,p^^) > a and ft.*(p2,P ') > Oi 
then for any S G [0, 1], 

g\Sp{ + (1 - S)p^,p~^) > f\Sp\ + (1 - S)pl,p~^)a, 

so that h^{Sp\ + (1 — S)pI,p~^) > a. One can now apply Glicksberg's [TT] theorem 
because the strategy sets are compact, convex, and the payoff functions are quasi- 
concave and continuous. □ 

Corollary 4.5. A profile po G A. is a Nash equilibrium of a Loomis game if and 
only if 

(4.2) po G argmin < max max |/i*(s*,p~') - /i*(p)| > . 

peA \^i=i....,N s»es» J 

''As far as we know, non-zero sum Loomis games are not considered in the literature. This 
model could be of interest in situations where there are populations with many players. A mixed 
strategy for a population describes the proportion of players in the population that uses some 
pure action. g'(p) is the non- normalized payoff of population i and /'(p) may be interpreted as 
the value of money for population i so that (p) = is the normalized payoff of population 

i. 

^Clearly, the lemma and its proof still hold in infinite games where the sets S" are convex- 
compact-metric and the functions and are continuous. The summation in the multi-linear 
extension should be replaced by an integral. 
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Proof. Clearly, po G ^ is an equilibrium of the Loomis game if and only if 

. / r 9ip\p-') 9Kp)\\ 
Po G are mm < max max < ^ --— > > . 

Using the quasi-linearity of the payoffs or Lemma l2.31 one deduces that: 



9'{P\P ') g'is\p ') 
max -r-Tzr. T = ™ax -r--—- r 

reA* f'{p\p-'') s'esi p{s\p-'') 

which is the desired result. □ 



Again, the min-max optimization problem (|4.2p is a particular instance of the 
MRF problem (|3.2p and so can be solved via the hierarchy of semidefinite relax- 
ations p. lip . Notice that in (|4.2p one has to minimize the supremum of rational 
functions (in contrast to the supremum of polynomials in (j4.ip '). 

4.3. Finite absorbing games. This subclass of stochastic games has been intro- 
duced by Kohlberg [^. The following formulas are established in [22]. It shows 
that absorbing games could be reduced to Loomis games. An iV-player finite ab- 
sorbing game is defined as follows. 

As above, there are N finite sets {S^, S^). There are two functions 5' : S ^ M 
and : S — > R associated to each player i G {1, N} and a probability transition 
function (7 : S ^ [0, 1]. 

The game is played in discrete time as follows. At each stage t = 1,2, if the 
game has not been absorbed before that day, each player i chooses (simultaneously) 
at random an action sj e S\ If the profile st — {si, s^) is chosen, then: 

(i) the payoff of player i is g*(st) at stage t. 

(ii) with probability 1 — q{st) the game is terminated (absorbed) and each player 
i gets at every stage s > t the payoff f^{st), and 

(iii) with probability q{st) the game continues (the situation is repeated at stage 
t+1). 

Consider the A-discounted game G(A) (0 < A < 1). If the payoff of player i at 
stage t is r'^{t) then its A-discounted payoff in the game is X^t^i ^(^ ^ A)*^^r*(t). 
Hence, a player is optimizing his expected A-discounted payoff. 

Let g"^ = g'^ X q and /* = /* x (1 — (7) and extend g*, /* and q multilinear ly to A 
(as above in Nash and Loomis games). 

A profile p G A is a stationary equilibrium of the absorbing game if (1) each 
player i plays iid at each stage t the mixed strategy until the game is absorbed 
and (2) this is optimal for him in the discounted absorbing game if the other players 
do not deviate. 

Lemma 4.6. A profile po d A is a stationary equilibrium of the absorbing game 
if and only if it is a Nash equilibrium of the Loomis game with payoff functions 
„ . Ag'(p)-Ki-A)/' (p) . _ -, ^ 

Proof See Laraki [23]. □ 

As shown in [23j . the min-max of a discounted absorbing game satisfies: 

Xg'{s\p-') + il-X) fHs\p-') 

V, = mm max — — — -—. -— . 

p-eA-'6'eS' Ag(s%p-*) -h (1 - g(s%p-*)) 
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Hence solving a finite absorbing game is equivalent to solving a Loomis game (hence 
a particular instance of the MRF problem ()3.2|) ) which again can be solved via 
the hierarchy of semidefinite relaxations (|3.1ip . Again one has to minimize the 
supremum of rational functions. 



5. Zero-sum polynomial games 

Let Ki C and K2 C M"^ be two basic and closed semi-algebraic sets (not 
necessarily with same dimension): 

(5.1) Ki := {xe :5,(a;) > 0, j-l,...,mi} 

(5.2) K2 := {xe K"' : hk{x) > 0, fc = l,...,m2} 

for some polynomials (gj) C R[a;i, . . . x„J and (hk) C ]R[a::i, . . . x^j]. 

Let P(Ki) be the set of Borel probability measures on Ki, i = 1, 2, and consider 
the following min-max problem: 



(5.3) P : J* = min max / / p(x, z) dn(x) dviz) 

MeP(Ki) i.eP(K2) JkJk, 

for some polynomial p G R[x, 2;]. 

If Ki and K2 are compact, it is well-known that 



(5.4) J* — min max / / p(x, z) dn(x) dviz) 

MeP(Ki) u&PCK^) JkJk, 



= max min / / p(x, z) dv(z) diilx), 
ueP{K2) A'eP(Ki) JkJk2 

that is, there exist fi* G P(Ki) and ly* £ P(K2) such that: 



(5.5) J* — / p{x, z) dfi* (x) dv* (z). 

Jk.2 Jki 

The probability measures /i* and v* are the optimal strategies of players 1 and 2 
respectively. 

For the univariate case n = 1, Parrilo [37] showed that J* is the optimal value 
of a single semidefinite program, namely the semidefinite program (7) in }37[ p. 
2858], and mentioned how to extract optimal strategies since there exist optimal 
strategies (^*,i/*) with finite support. In [37j the author mentions that extension 
to the multivariate case is possible. We provide below such an extension which, in 
view of the proof of its validity given below, is non trivial. The price to pay for this 
extension is to replace a single semidefinite program with a hierarchy of semidefinite 
programs of increasing size. But contrary to the polynomial optimization case in 
e.g. [25, proving convergence of this hierarchy is more delicate because one has 
(simultaneously in the same SDP) moment matrices of increasing size and an s.o.s.- 
representation of some polynomial in Putinar's form (j2.3p with increasing degree 
bounds for the s.o.s. weights. In particular, the convergence is not monotone 
anymore. When we do n = 1 in this extension, one retrieves the original result of 
Parrilo [37], i.e., the first semidefinite program in the hierarchy (|5.9p reduces to (7) 
in [37l p. 2858] and provides us with the exact desired value. 
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Semidefinite relELxations for solving P. With p e M[a;,z] as in (j3.2|) . write 
(5.6) pix,z) = ^ pa{x) z"' with 

Pa{x) = ^ PapX^, \a\ < 4 

where dz is the total degree of p when seen as polynomial in R[z]. So, let pap :— 
for every f3 G N"^ whenever \a\ > dz- 

Let Tj := [deg(7j/2], for every j — 1, . . . ,mi, and consider the following semi- 
definite program: 



min A 



s.t. 



(5.7) <^ 



/3eN"i fe=i 
Md{y) h 

A^<i-r,(gj,y) ^ 0, j = i,...,mi 

2/0 = 1 

Z'= ^0, fc = 0,1,... 7712, 



where y is a finite sequence indexed in the canonical basis (x") of R [a;] 2d- Denote 
by its optimal value. In fact, with /iq = 1 and Py G R[z] defined by: 



(5.8) 



z ^ py{z) := ^ Y Papyp \ z , 



the semidefinite program (|5.7p has the equivalent formulation: 



min A 



(5.9) 



s.t. A - py[-) = y^gfc hk 



k=0 



Md{y) h 

Md-r, [g-j , y) ^ 0, j = 1, . . . , mi 
yo = 1 

(Tfc G E[z]; : deg CTfc + deg/ifc < 2(i, fc = 0, 1, . . . ,TO2, 



where the first constraint should be understood as an equality of polynomials. 
Observe that for any admissible solution (y, A) and py as in 



(5.10) 



A > max{py(z) : z G K2}. 
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Similarly, with p as in (|3.2p . write 

(5.11) p{x,z) — ^ pa{z)x°' with 

QgN"l 

Pa{z) = ^ Pa(3z'^, \a\ < dx 

where dx is the total degree of p when seen as polynomial in So, let pap :~ 

for every (3 G N"^ whenever \a\ > dx- 

Let Ik '■— [deg hk/2~\ , for every k — I, . . . , TO2, and with 

(5.12) X ^ Py{x) X! H Pa/3 2//3 a;", 
consider the following scmidcfinite program (with go = 1): 



(5.13) 



max 7 

mi 

s.t. py(-) - 7 = J2'^3 9j 

Md{y) h 
Md-ijhk,y) h 0, A 

yo = 1 



,m2 



aj eY.[x]; degaj + deg gj < 2d, j = 0, 1, . . . , toi. 

where y is a finite sequence indexed in the canonical basis (z") of R[z]2(i- Denote 
by 7^ its optimal value. In fact, (|5.13p is the dual of the semidefinite program ([57 



Observe that for any admissible solution (y,7) and py as in (|5.12p . 
(5.14) 7 < mm{py{x) : x e Ki}. 



Theorem 5.1. Let P he the min-max problem defined in iS. 2\) and assume that 
both Ki and K2 are compact and satisfy Putinar's property (see Definition \2.1]) . 
Let X*^ and 7^ he the optimal values of the semidefinite program 115. 9\) and 115. 1S\) 
respectively. Then J* and ^ J* as d —> 00. 

We also have a test to detect whether finite convergence has occurred. 

Theorem 5.2. Let P he the min-max problem defined in iS.S^) . 

Let AJ he the optimal value of the semidefinite program I15.9\) . and suppose that 
with r :— maxj=i^..._,„j rj, the condition 

(5.15) rankAf<j_^(y) = rankAf<j(y) (=: si) 

holds at an optimal solution (y, A, (Tfc) of i5.9\) . 

Let 7(* he the optimal value of the semidefinite program 115. 13\) . and suppose that 
with r :— max^^i ...^„i2 h, the condition 

(5.16) rankAft_,(y') = rankAft(y') (=: S2) 
holds at an optimal solution (y',7,crj) of ^5.13]) . 
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// X*i = 7(* then X* — 7^* = J* and an optimal strategy for player 1 ( resp. player 
2) is a probability measure fj,* (resp. v* ) supported on Si points 0/ Ki (resp. S2 
points of K2J. 

For a proof the reader is referred to ^ 

Remark 5.3. In the univariate case, when Ki,K2 are (not necessarily bounded) 
intervals of the real line, the optimal value J* = (resp. J* =7^) is obtained by 
solving the single semidefinite program (|5.9p (resp. (|5.13p ) with d = c?o, which is 
equivalent to (7) in Parrilo 37, p. 2858]. 



6. Zero-sum polynomial absorbing games 

As in the previous section, consider two compact basic semi-algebraic sets Ki C 
R"i, K2 C M"^ and polynomials 5, / and g : Ki x K2 ^ [0, 1]. Recall that 
P(Ki) (resp. P(K2)) denotes the set of probability measures on Ki (resp. K2). 
The absorbing game is played in discrete time as follows. At stage t — 1,2,... 
player 1 chooses at random xt 6 Ki (using some mixed action /it G P(Ki)) and, 
simultaneously. Player 2 chooses at random yt € K2 (using some mixed action 

eP(K2)). 

(i) player 1 receives g{xt,yt) at stage t; 

(ii) with probability 1 — q{xt,yt) the game is absorbed and player 1 receives 
/ [xt,yt) in all stages s > t; 

and 

(iii) with probability q(xt,yt) the game continues (the situation is repeated at 
step t + 1). 

If the stream of payoffs is r(<), < = 1, 2, the A-discounted-payoff of the game 
isEt=i^A(l-ArMi)-^ 

Let g = g X q and / = / x (1 — g) and extend g, f and q multilinearly to 
P(Ki) xP(K2). 

Player 1 maximizes the expected discounted-payoff and player 2 minimizes that 
payoff. Using an extension of the Shapley operator [45] one can deduce that the 
game has a value v (A) that uniquely satisfies, 

v(\) = max min / ( Ao + (1 — A)i;(A)p + (1 — A) f ) dii^v 

= min max / ( Ag -f (1 — A)ti(A)n + (1 — A) f ) dn®v 
^eP(K.)MGP(Ki)7e V > 

with 8 := Ki x K2. As in the finite case, it may be shown [23] that the problem 
may be reduced to a zero-sum Loomis game, that is: 



(6.1) V\X) = max mm — mm max 

MeP(Ki) iyeP(K2) l^Qd^i^v i/eP(K2) MeP(Ki) l^QdpL^v 

where 

{x,y)^P(x,y) := Ag(.T, y) + (1 - A)/(.t, y) 
{x,y)^Q[x,y) := \q(x,y) ^ \ - q{x,y) 
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Or equivalently, as it was originally presented by Loomis ^32j, v (A) is the unique 
real t such that 



— max min 



mm mm 



{P{x, y) -tQ{x, y)) dii{x)dv{y) 

{P{x, y)-t Q{x, y)) d^{x)dv{y) 
defined by: 

{P{x, y) - tQ{x, y)) d^i,{x)dv{y) 



Actually, the function s : R - 

t — > sit) :— max min 
MeP(Ki)i.eP(K2) 

is continuous, strictly decreasing from +00 to — cxd as t increases in (— 00, +c)o). 

In the univariate case, if Ki and K2 are both real intervals (not necessarily 
compact), then evaluating s{t) for some fixed t can be done by solving a single 
semidefinite program; see Remark l5.3l Therefore, in this case, one may approximate 
the optimal value s* (= s{t*)) of the game by binary search on t and so, the problem 
can be solved in a polynomial time. This extends Shah and Parrilo [44] . 



7. Conclusion 

We have proposed a common methodology to approximate the optimal value 
of games in two different contexts. The first algorithm, intended to compute (or 
approximate) Nash equilibria in mixed strategies for static finite games or dynamic 
absorbing games, is based on a hierarchy of semidefinite programs to approximate 
the supremum of finitely many rational functions on a compact basic semi-algebraic 
set. Actually this latter formulation is also of self-interest in optimization. The 
second algorithm, intended to approximate the optimal value of polynomial games 
whose action sets are compact basic semi-algebraic sets, is also based on a hierarchy 
of semidefinite programs. Not surprisingly, as the latter algorithm comes from a 
min-max problem over sets of measures, it is a subtle combination of moment and 
s.o.s. constraints whereas in polynomial optimization it is entirely formulated either 
with moments (primal formulation) or with s.o.s. (dual formulation). Hence the 
above methodology illustrates the power of the combined moment-s.o.s. approach. 
A natural open question arises: how to adapt the second algorithm to compute 
Nash equilibria of a non-zero-sum polynomial game? 



8. Appendix 

8.1. Proof of Theorem 15.11 We first need the following partial result. 

Lemma 8.1. Let (y'*)d he a sequence of admissible solutions of the semidefinite 
program |5. 7| ). Then there exists y G and a subsequence (di) such that y"^' y 
pointwise as i ^ 00, that is, 

(8.1) lim yi^ = Va G N". 

i — >oo 

The proof is omitted because it is exactly along the same lines as the proof of 
Theorem 13.41 as among the constraints of the feasible set, one has 

2/^ = 1, Maiy^)hO, Maig,, y^) h 0, J ^ I, ■ ■ ■ ,mi. 
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Proof of Theorem \5.1\ Let /i* G P(Ki), i/* £ P{K.2) be optimal strategies of player 
1 and player 2 respectively, and let y* — (y*) be the sequence of moments of 
(well-defined because Ki is compact). Then 



J* = max / / p(x, z)dfj,* (x) dv{z) 



= max / > > PaS / dn*(x) z'^dviz) 



max 



Py'{z)dv{z) 
max {py(z) : z G K2} 



= min{A : A -py.(-) = (To + y'cTfe ft,fe; (o-j)"l''n C 
with z 1-^- py. (z) defined in (jS.Sp . Therefore, with e > fixed arbitrary, 

(8.2) j*_py.(.)+e ^ ^^ + ^^^/,,, 

fe=i 

for some polynomials {a%) C of degree at most 2d\. So (y*, J* + e, cr|) is an 
admissible solution for the semidefinite program (|5.9p whenever d > maxj Tj and 
(i > + maxfe 1^ , because 

(8.3) 2(i>degCTQ; 2d > degtr^ + deg/i/c, fc = l,...,TO2. 
Therefore, 



(8.4) \*i < J* + e, Vd > := max 



max Tj , dl + max 



Now, let (y'',Ad) be an admissible solution of the semidefinite program ()5.9p 
with value A^ < A^ + 1 /d. By Lemma 18.11 there exists y G K°° and a subsequence 
(di) such that y'^' y pointwise, that is, (|8.ip holds. But then, invoking (|8.ip 
yields 

Md(y) ^ and Afd(.9j, y) ^0, Vj = 1, . . . , mi; d = 0, 1, . . . 
By Theorem 12. 2[ there exists fi E P(Ki) such that 



Ha = djji, Va e 



On the other hand. 



J* < max / / p{x, z)djl(x) diy{z) 
•yeP(K2)JK2 \Jki J 

= max{py(z) : z e K2} 

z 

= min{A : A - Py(-) = ctq + ^ Cfc hk; {cr^T^ia C S[z]} 

fc=i 
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with 

Next, let p :— max2gK2 (-z) (hence p > J*), and consider the polynomial 

aeti" J 

It has same degree as py, and by (|8.ip . \\py{-) — Pydi (•)|| — > as i ^ oo. 

Hence, max2gK2 Py-^i (z) p as i oo, and by construction of the semidefinite 
program (|5.9p . A^^ > max^gKs Py^iiz)- 

Therefore, AjJ. > p — e for all sufficiently large i (say di > d^) and so, A|J. > J* — e 
for all di > d^. This combined with AjJ. < J* + e for all di > dl, yields the desired 
result that limi^oo A^J. — J* because e > fixed was arbitrary; 

Finally, as the converging subsequence (r^) was arbitrary, we get that the entire 
sequence (A^) converges to J*. The convergence 7^ J* is proved with similar 
arguments. □ 

8.2. Proof of Theorem 15.21 By the flat extension theorem of Curto and Fialkow 
(see e.g. [26 ), y has a representing si-atomic probability measure p* supported on 
Ki and similarly, y' has a representing S2-atomic probability measure v* supported 
on K2. But then from the proof of Theorem 15. 11 



J* < max / / p{x, z)dp* (x) dtp{z) 



max 



IX / py[z)dip{z) = max{py(z) : z G K2} < X*^. 
K2) JK2 ^ 



> min / / p{Xt z)dv* {z) \ d'il}{x) 
,AeP(Ki)JKi Vjk2 



mm 

1/jeP 



in / py/(a;)dV(a;) = min{py/(a;) : x G Ki} > 7t*, 
(Ki)Jki ^ 



and so as A)^ = 7^* one has J* = AjJ = 7t*. This in turn implies that p* (resp. v*) 
is an optimal strategy for player 1 (resp. player 2). □ 
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